Grounding Visual Representations with Texts for Domain Generalization

Reducing the representational discrepancy between source and target domains is a key component to maximize model generalization. In this work, we advocate for leveraging natural language supervision domain generalization task. We introduce two modules ground visual representations with texts containing typical reasoning of humans: (1) Visual Textual Joint Embedder (2) Explanation Generator. The former learns image-text joint embedding space where can high-level class-discriminative information into model. latter leverages an explainable generates explanations justifying rationale behind its decision. To best our knowledge, first work leverage vision-and-language cross-modality approach Our experiments newly created CUB-DG benchmark dataset demonstrate that be successfully used domain-invariant improve Furthermore, in large-scale DomainBed benchmark, proposed method achieves state-of-the-art results ranks 1st average performance five multi-domain datasets. codes are available at https://github.com/mswzeus/GVRT.